We present a novel analysis of the expected risk of weighted majority vote in multiclass classification. The analysis takes correlation of predictions by ensemble members into account and provides a bound that is amenable to efficient minimization, which yields improved weighting for the majority vote. We also provide a specialized version of our bound for binary classification, which allows to exploit additional unlabeled data for tighter risk estimation. In experiments, we apply the bound to improve weighting of trees in random forests and show that, in contrast to the commonly used first order bound, minimization of the new bound typically does not lead to degradation of the test error of the ensemble.

Additional Feedback: Below one can find my questions and suggestions for authors. I rated this article as 7 only on the condition that my questions / suggestions will be treated by the authors. "PAC-Bayesian generalization bound on confusion matrix for multi-class classification." UPDATE: I thank the authors for providing the reply. I agree with their points.

The authors proposes a new PAC-Bayes bound on majority vote classifiers (such as, but not restricted to, classifiers that comes from ensemble methods). Contrarily to most of known PAC-Bayes bounds, their proposed bound do not only take into account the so called Gibbs risk (which can be view as the average quality of the voters one which the majority vote is build), but also on some second order information (that quantifies how much the classifiers of the majority votes are decorrelated on their errors. A similar attempt has been proposed by Lacasse et al. (2007) & Germain et al. (2015). Their approach is nevertheless more general, simpler and seems to leads to tighter majority vote bounds. Most of the learning algorithms can be analysed through the PAC-Bayesian theory.

We present a novel analysis of the expected risk of weighted majority vote in multiclass classification. The analysis takes correlation of predictions by ensemble members into account and provides a bound that is amenable to efficient minimization, which yields improved weighting for the majority vote. We also provide a specialized version of our bound for binary classification, which allows to exploit additional unlabeled data for tighter risk estimation. In experiments, we apply the bound to improve weighting of trees in random forests and show that, in contrast to the commonly used first order bound, minimization of the new bound typically does not lead to degradation of the test error of the ensemble.